Integration Guide ABB (OmniCore)

Note: We strongly recommend reading the Robot communication overview before this integration guide.

Note 2: Always use the integration guide that comes with the latest version of the Robot module. You can download the Robot module from the official Photoneo website.

This guide describes the Robot module 1.12.0 for ABB OmniCore controllers. It works with Bin Picking Studio (BPS) 1.12.

Contents

• 1 Prerequisites

• 2 Robot controller setup

  • 2.1 Controller configuration

    • 2.1.1 Network configuration

    • 2.1.2 Firewall configuration

  • 2.2 Robot module installation

    • 2.2.1 Loading the Robot module files

    • 2.2.2 Creating the Robot State Server task

    • 2.2.3 Automatic loading of modules

    • 2.2.4 Loading the example program

• 3 Robot module

  • 3.1 Robotic API

    • 3.1.1 Connection procedures

    • 3.1.2 Communication procedures

    • 3.1.3 Bin picking procedures

    • 3.1.4 Auxiliary procedures

    • 3.1.5 Info data

    • 3.1.6 Global variables

    • 3.1.7 Wizard procedures

  • 3.2 Example programs

    • 3.2.1 Basic bin picking example

    • 3.2.2 Optional utilities

  • 3.3 Error handling

• 4 Running the basic bin picking example program

  • 4.1 Prerequisites

  • 4.2 Bin picking routine execution settings

  • 4.3 Reteach the robot poses

  • 4.4 Runtime

• 5 Robot State Server

• 6 Calibration

• 7 Troubleshooting

• 8 Version and brand ID

• 9 Migration

  • 9.1 BPS 1.11.x -> BPS 1.12.x

  • 9.2 Robot module update

1 Prerequisites

The Robot module runs on OmniCore controllers with RobotWare 7.x.

The following RobotWare options are required:

• (3119-1) RobotStudio Connect - provides the socket communication used by the Robot module

• (3114-1) Multitasking - runs the Robot State Server in its own background task

Go to Control Panel (hardware key on the FlexPendant) -> Info to check that your system has these options:

Note: To install the Robot module, use a RobotStudio version that supports your RobotWare 7 version.

Note: RobotWare 7 has a controller firewall that blocks RAPID sockets by default. Open it as described in Firewall configuration.

2 Robot controller setup

2.1 Controller configuration

2.1.1 Network configuration

The vision controller connects to the WAN port of the OmniCore controller. Set the IP address of the WAN port to the same subnet as the vision controller.

Turn on the OmniCore controller and wait for the initialization screen. Then open Settings:

Choose the Network option:

The Network Settings window opens. Use the Touch Panel Keypad to set the IP address of the OmniCore WAN port:

Select Apply and restart the controller (Control Panel -> Logout/Restart -> Restart) to apply the new settings:

2.1.2 Firewall configuration

RobotWare 7 blocks the RapidSockets network service by default. Without this service, the Robot module cannot connect to the vision controller, and the Robot State Server cannot accept a connection.

In RobotStudio, go to Configuration -> Communication -> Firewall Manager -> RapidSockets and set Enable on Public Network and Enable on Private Network to Yes. Restart the controller.

The Robot module checks this setting itself. If RapidSockets is disabled, the module writes event log message 4801 (Communication not enabled in the robot firewall!) and stops:

• pho_wait_for_server checks the private network if the vision controller IP address starts with 192.168.125., and the public network for any other address.

• The Robot State Server stops if RapidSockets is disabled on both networks.

Note: The check only reports the problem. You must always open the firewall manually in RobotStudio.

2.2 Robot module installation

2.2.1 Loading the Robot module files

Transfer the Robot module files to the robot controller. You can use a USB stick or RobotStudio (described below).

Set the network settings of your PC to DHCP and connect the PC to the Management Port of the OmniCore controller with an Ethernet cable. Then start RobotStudio.

To connect directly to the robot controller, select the Controller tab and choose the One Click Connect option:

Go to the Controller tab and select the File Transfer tool from the menu:

The Robot module (folder rapid in the downloaded package) contains these files:

File	Load into task	Purpose
PHO_BPS_BASE.sysx	all tasks	System module with the communication driver, the Robotic API, constants and global variables. Do not edit.
PHO_BPS_Customer_Definitions.modx	T_ROB1	Gripper procedures, speed settings and the bin picking routine. Edit the gripper procedures and speeds here.
PHO_BPS_State_Server.modx	PHO_STATE_SERVER	Robot State Server - sends the robot joint states and tool pose to the vision controller.
PHO_BPS_Wizard.modx	T_ROB1 (optional)	High-level wrapper procedures to build a program from a few calls - see Wizard procedures.
example_programs/PHO_BPS_*.modx	T_ROB1 (one at a time)	14 example programs - see Example programs.
optional_utilities/CollisionDetect.modx, optional_utilities/CycleTimeChecker.sysx	T_ROB1 (optional)	Collision detection with recovery and cycle time monitoring - see Optional utilities.

Copy the files (and the example program you want to use) to the folder HOME:/Photoneo/ in the controller file system:

Note: The screenshots in this chapter show an older version of the Robot module, with old file names. PhotoneoCommon.sys is now PHO_BPS_BASE.sysx, CustomerDefinitions.mod is now PHO_BPS_Customer_Definitions.modx, PhotoneoStateServer.mod is now PHO_BPS_State_Server.modx and MainModule.mod is now PHO_BPS_Basic.modx. The settings shown are still valid.

2.2.2 Creating the Robot State Server task

The Robot State Server runs in its own background task and reports the current state of the robot to the vision controller (see Robot State Server).

If your motion task is not called T_ROB1 (for example in MultiMove systems with T_ROB1 and T_ROB2), open PHO_BPS_State_Server.modx and change the constant robot_task_id to the name of the motion task that does the bin picking.

In RobotStudio, select the Request Write Access option on the Controller tab. Then select the Grant Access option on the FlexPendant. In the left pane, open Configuration -> Controller, right-click the Task item and select New Task…:

Set up the task as shown in the figure below - Task: PHO_STATE_SERVER, Type: Semistatic, Main Entry: pho_state_server_main, TrustLevel: SysStop:

2.2.3 Automatic loading of modules

Configure the controller to load the Robot module automatically at startup. Create three entries in Configuration -> Controller -> Automatic Loading of Modules:

File	Settings
HOME:/Photoneo/PHO_BPS_BASE.sysx	All Tasks: Yes
HOME:/Photoneo/PHO_BPS_State_Server.modx	Task: PHO_STATE_SERVER
HOME:/Photoneo/PHO_BPS_Customer_Definitions.modx	Task: T_ROB1

Note: PHO_BPS_BASE.sysx must be loaded in both tasks - T_ROB1 and PHO_STATE_SERVER. The Robot State Server uses its brand ID, helper functions and the shared flag pho_connecting_to_server.

Right-click Automatic Loading of Modules and select New Automatic Loading of Modules:

Entry for the Robot State Server module:

Entry for the base system module (All Tasks: Yes):

Entry for the customer definitions module:

2.2.4 Loading the example program

Load one example program into the T_ROB1 task. Every example has its own main routine, so you can load only one at a time. Start with PHO_BPS_Basic.modx.

Right-click the T_ROB1 task and select Load Module or Load Module From Controller:

Restart the robot controller to apply all configuration changes. The RAPID tree should now look like the picture below (with the old module names):

The robot controller is now configured to work with BPS.

3 Robot module

The Robot module is a set of RAPID modules that you can use in existing applications. The robot program calls the procedures of the Robotic API, and the module handles the TCP communication with the vision controller.

3.1 Robotic API

Note: We strongly recommend reading the Photoneo robotic API and Action requests before this section.

All procedures of the Robotic API are declared in PHO_BPS_BASE.sysx. Do not edit them. Every procedure that communicates with the vision controller writes the result to the global variables PHO_ERR_CODE (error code) and PHO_OCCURRED_ERR (TRUE if an error occurred).

3.1.1 Connection procedures

Connection procedure

Description / Usage

Connect to Action Request Server pho_wait_for_server ( string server_ip, \num port_number, \num wait_time, \switch sys_stop | switch sys_restart )

Description Opens the connection to the Action Request Server on the vision controller and sends the brand ID. It also resets the internal state of the module (request flags, trajectory buffer, Info data variables, pho_debug). Before connecting, it checks that RapidSockets is enabled in the controller firewall (see Firewall configuration). Input parameters: server_ip - IP address of the vision controller port_number - port of the Action Request Server [optional - default 11003, we recommend omitting it] wait_time - connection timeout in seconds [optional - default 3600 s] sys_stop | sys_restart - what to do if the connection fails [optional]. sys_stop writes event log message 4800 and stops the program (EXIT). sys_restart writes a message and restarts the program from main (ExitCycle). Without a switch, the RAPID error is raised to the calling routine. Output: PHO_ERR_CODE = 5 (Timeout) or 3 (Communication error) if the connection fails. Usage Call it once at the beginning of the program, and again to reconnect after a communication error. You can send requests only after the connection is established. pho_wait_for_server "192.168.1.1", \wait_time:=20, \sys_restart;

3.1.2 Communication procedures

Note: See Action requests for the detailed behavior of each request.

Synchronous procedures send the request and wait for the response. Asynchronous procedures only send the request. A response receiving procedure collects the response later. With a hand-eye sensor, keep the robot still until the scan response arrives. With an extrinsic sensor, the robot can move in the meantime if it stays out of the scanned scene. The only exception is dynamic MultiView, where the sensor scans while the robot moves.

All procedures set PHO_ERR_CODE and PHO_OCCURRED_ERR. Procedures declared as FUNC num also return a value (the error code unless stated otherwise).

Procedure	Request ID	Inputs	Outputs
pho_request_init (synchronous)	4	jointtarget pho_start_bin_picking_pose - Start pose, jointtarget pho_end_bin_picking_pose - End pose, num vision_system_id, \num wait_time - response timeout [s, optional, default 3600]	error code
pho_request_scan (asynchronous)	1	num vision_system_id	response by pho_wait_for_scan_completion
pho_trigger_scan (asynchronous)	28	num vision_system_id, \robtarget pose - pose of the scanner for hand-eye multiview [optional]	response by pho_wait_for_scan_completion
pho_reuse_scan	29	num vision_system_id, \robtarget pose [optional]	response by pho_wait_for_scan_completion
pho_request_trajectory (asynchronous)	2	num vision_system_id	response by pho_receive_trajectory
pho_request_send_pick_failed (synchronous)	7	num vision_system_id	error code
pho_request_object_pose (synchronous)	8	num vision_system_id	pho_object_pose (robtarget, mm), Info data 4-7
FUNC num pho_change_env_state (synchronous)	15	num state_id - scene state ID	returns the error code
pho_request_get_vs_status (synchronous)	21	num vision_system_id	pho_num_localized, pho_num_ready, pho_process_state
FUNC num pho_change_bbox (synchronous)	32	num vision_system_id, num bbox_id - bounding box ID	returns the error code
FUNC num pho_request_calib_add_point (synchronous)	5	-	returns the error code
FUNC num pho_request_calib_start (synchronous)	25	num solution_id, num vision_system_id	returns the error code
FUNC num pho_request_calib_save (synchronous)	27	-	returns the error code; pho_calib_acc, pho_camera_pose
FUNC num pho_request_calib_stop (synchronous)	26	-	returns the error code
pho_request_change_solution (synchronous)	9	num solution_id	error code
FUNC num pho_request_solution_start (synchronous)	10	num solution_id	returns the error code
FUNC num pho_request_solution_stop (synchronous)	11	-	returns the error code
FUNC num pho_request_get_running_sol (synchronous)	12	-	returns the ID of the running solution (-1 if the request could not be sent)
FUNC num pho_request_get_available_sol (synchronous, deprecated)	13	-	returns the error code
FUNC num pho_comm_check (synchronous)	100	-	returns the error code

Response receiving procedures

Procedure	Inputs	Outputs
pho_wait_for_scan_completion	\num wait_time - response timeout [s, optional, default 3600]	error code of the last scan, trigger scan or reuse scan request
pho_receive_trajectory	\num wait_time - response timeout [s, optional, default 3600]	trajectory in pho_trajectory_buffer, Info data 1-7, pho_pick_allowed = TRUE if no error

Notes:

• The module remembers which requests are waiting for a response. If a request of the same kind is still pending, a new request first collects the old response. For example, pho_request_trajectory first waits for the pending scan.

• pho_receive_trajectory stops the program (event log message 4800 Program flow error) if no trajectory was requested. Always call pho_request_trajectory first.

• Synchronous requests without a wait_time parameter wait up to 3600 s for the response.

• The pose of pho_trigger_scan / pho_reuse_scan is sent as X, Y, Z [mm] and a quaternion. The examples pass the current pose of the tool pho_tool.

• pho_request_bin_locator (request ID 3) exists in the module but is not supported by BPS.

3.1.3 Bin picking procedures

These procedures are in PHO_BPS_Customer_Definitions.modx. Adapt the gripper procedures and binpicking_settings to your application.

Procedure	Description / Usage
gripper_attach	Implement the command that grips the object. It is executed when the Grasping method sends the Attach gripper command.
gripper_detach	Implement the command that releases the object. It is executed when the Grasping method sends the Detach gripper command. The placing part of the main program usually calls it too.
gripper_user_1, gripper_user_2, gripper_user_3	User-defined gripper commands. They are executed when the Grasping method sends the User 1, User 2 or User 3 gripper command.
binpicking_settings	Sets the speed of each trajectory of the bin picking routine (array pho_speeddata). pho_bin_picking calls it automatically. See Bin picking routine execution settings.
pho_bin_picking	Executes the received bin picking routine - all trajectories and gripper commands in the order sent by the vision controller. Do not edit. Call it after pho_receive_trajectory, with the robot at the Start pose. At the end, the robot is at the End pose with the object in the gripper. If no trajectory was received successfully (pho_pick_allowed = FALSE), it stops the program with event log message 4800 Program flow error.
pho_bin_picking_simple	Simple alternative that calls pho_receive_trajectory itself. It executes trajectories 1-4 with fixed speeds, zones and gripper calls. Use it as a template for your own routine.
pho_execute_trajectory (num index, \speeddata speed, \zonedata zone)	Executes one trajectory of the received routine with MoveAbsJ. Intermediate points use zone z20. The last point uses fine if the trajectory is of type fine, otherwise zone (default z20). The default speed is pho_speeddata{index}.
execute_gripper(num index)	Calls the gripper procedure for the received gripper command number index.

3.1.4 Auxiliary procedures

Procedure	Description
pho_enable_debug	Sets pho_debug to TRUE. The module then writes every request, response and operation to the FlexPendant. pho_wait_for_server turns the debug output off, so call pho_enable_debug after you connect.
FUNC string pho_err_info(num code)	Returns a short text for an error code (for example "NO PART FOUND"). Unknown codes return "undefined error <code>".
FUNC string pho_req_info(num id)	Returns the name of a request ID.
FUNC string pho_gripper_info(num cmd)	Returns the name of a gripper command.
FUNC jointtarget deg2rad_jointtarget(jointtarget), FUNC jointtarget rad2deg_jointtarget(jointtarget)	Convert all axes of a joint target between degrees and radians.
pho_check_firewall_settings(string server_ip), FUNC bool pho_firewall_enabled(\switch private | switch ionet | switch public)	Firewall check used by pho_wait_for_server (see Firewall configuration).

3.1.5 Info data

Values returned by the requests (other than error codes, poses and trajectories) are Info data. The module stores each value in the variable listed below. Each variable keeps its value from the last response that contained it, so always check PHO_ERR_CODE first.

Key	Info data	Variable	Unit in robot
1	Tool point invariance	pho_tool_point_invariance	index
2	Gripping point ID	pho_gripping_point_id	index
3	Gripping point invariance	pho_gripping_point_invariance	index
4	Dimension X	pho_dimension_x	mm
5	Dimension Y	pho_dimension_y	mm
6	Rotation Z	pho_dimension_rot	deg (converted from mrad)
7	Neural network label	pho_nn_label	integer (-1 = not received)
8	Max Z height	pho_max_z_height	mm (reserved - not sent by BPS 1.12)
9	Tilt	pho_tilt	mrad, as sent (reserved - not sent by BPS 1.12)
10	Calibration accuracy	pho_calib_acc	µm
11	Running solution ID	current_solution_id (internal) - read it as the return value of pho_request_get_running_sol	integer
12	Number of localized objects	pho_num_localized	integer
13	Number of ready objects	pho_num_ready	integer
14	Processing state	pho_process_state	0 / 1

pho_wait_for_server resets keys 1-3 and 7 to -1 and keys 4-6, 8 and 9 to 0.

3.1.6 Global variables

In addition to the Info data variables, PHO_BPS_BASE.sysx declares these PERS variables:

Variable	Type	Meaning
PHO_ERR_CODE	num	Error code of the last request / response.
PHO_OCCURRED_ERR	bool	TRUE if the last request / response ended with an error.
pho_trajectory_buffer	jointtarget{10,100}	Received trajectories [deg] - up to 10 trajectories with up to 100 points each.
pho_trajectory_sizes	num{10}	Number of points in each trajectory.
pho_trajectory_fine_cnt	bool{10}	TRUE if the last point of the trajectory must be reached with fine.
pho_operations, pho_num_of_operations	num{20}, num	Order of the received operations (trajectories and gripper commands) and their number.
pho_gripper_command	num{10}	Received gripper commands (1 attach, 2 detach, 3-5 user 1-3).
pho_object_pose	robtarget	Object pose from pho_request_object_pose [mm, quaternion].
pho_camera_pose	robtarget	Calibration result (camera pose) from pho_request_calib_save.
pho_speeddata, pho_zonedata	speeddata{10}, zonedata{10}	Speed and zone of each trajectory. pho_wait_for_server initializes them to v300 / fine. binpicking_settings then sets the speeds.
pho_pick_allowed	bool	TRUE after a trajectory is received successfully. pho_bin_picking resets it.
pho_connecting_to_server	bool	Shared with the Robot State Server. Pauses the Robot State Server while pho_wait_for_server connects.
pho_debug	bool	Debug output to the FlexPendant (see pho_enable_debug).
pho_available_solutions	num{100}	Belongs to the deprecated Get available solutions request - not used.
sPho_IP_StateServer (in PHO_BPS_State_Server.modx)	string	IP address that the Robot State Server listens on. Empty (default) = automatic. See Robot State Server.

3.1.7 Wizard procedures

PHO_BPS_Wizard.modx contains high-level procedures. Each procedure is a complete step of a bin picking application. You can build a short program from these calls only (for example, with block programming on the FlexPendant). _EX procedures are for a stationary (extrinsic) scanner. _HE procedures are for a robot-mounted (hand-eye) scanner. The module uses its own tool tPho_Tool and home pose pPho_home_pose.

Procedure	Description
PHO_INIT_EX(string VC_IP, num VisionID, robtarget HomePose, robtarget Start_Bin_Picking, robtarget End_Bin_Picking)	Connects (stops the program if the connection fails), sends the Initialization request, moves to the home pose and starts the first scan.
PHO_PICK(num VisionID)	Waits for the scan, requests and receives the trajectory, moves to the start pose and executes pho_bin_picking. Retries after errors (see PHO_ERR_HANDLING).
PHO_PLACE_EX(robtarget PlacePosition, num OffsZ)	Starts the next scan and then places the object: approaches to OffsZ mm above the place position, calls gripper_detach and retreats.
PHO_INIT_HE(...)	Same parameters as PHO_INIT_EX. Connects, initializes and moves to the home pose. Does not start a scan.
PHO_SCAN_HE(robtarget ScanPose, num Vision_id, \num Wait_time)	Moves to the scan pose, waits Wait_time s (optional) and starts the scan.
PHO_PLACE_HE(robtarget PlacePosition, num OffsZ)	Places the object. Does not start a scan - call PHO_SCAN_HE for the next cycle.
PHO_ERR_HANDLING	Called by PHO_PICK. Stops the program on communication errors (3, 4, 5) or after more than 3 errors in a row. Otherwise re-initializes if needed (203, 1), clears the error and starts a new scan.

Note: PHO_INIT_EX / PHO_INIT_HE copy only the TCP frame of tool0 into tPho_Tool. Set the load data of tPho_Tool for your gripper.

3.2 Example programs

The example programs are in the folder example_programs. Each one is a complete module for the T_ROB1 task. Their error handling is only a demonstration - adapt it to your application.

All examples connect to 192.168.1.1 and use the tool pho_tool (declared in the example as a TASK PERS tooldata with zero TCP). Change both for your cell. All poses are initialized to an unreachable value (99999.0 or 9E+09), so the robot does not move until you reteach them.

Example program	Purpose
PHO_BPS_Basic.modx	Basic bin picking loop: initialization, scan, trajectory, pick, place. The next scan starts while the object is being placed.
PHO_BPS_Multiple_Vision_Systems.modx	Two vision systems, each with its own Start / End poses. The program switches between them after every pick.
PHO_BPS_Reuse_Scan.modx	Same as Multiple vision systems, but vision system 2 uses pho_reuse_scan (localize on the last scan) instead of a new scan.
PHO_BPS_Change_Solution.modx	After every pick, switches between solutions 1 and 2 with pho_request_change_solution and initializes again.
PHO_BPS_Change_Environment.modx	After every pick, switches between scene states 1 and 2 with pho_change_env_state and initializes again.
PHO_BPS_Change_BBox.modx	After every pick, switches between bounding boxes 1 and 2 with pho_change_bbox (no new initialization needed).
PHO_BPS_Get_Status.modx	Basic loop that reads the vision system status (pho_request_get_vs_status) five times after each scan.
PHO_BPS_Get_Object_Pose.modx	Localization without path planning: pho_request_object_pose, then the robot moves to pho_object_pose.
PHO_BPS_Hand_Eye.modx	Basic loop for a robot-mounted scanner. The scan starts from the home pose, so the bin must be in view from this pose.
PHO_BPS_Multiview_Static.modx	Hand-eye multiview: the robot stops at poses p1-p3 and calls pho_trigger_scan at each pose. Then pho_request_scan localizes on the merged point cloud.
PHO_BPS_Multiview_Dynamic.modx	Hand-eye multiview while the robot moves: a timer interrupt calls pho_trigger_scan every frequency seconds along p1-p7. Then the program calls pho_request_scan. A distance-based trap routine is included as an alternative.
PHO_BPS_Comm_Check.modx	Connects and sends pho_comm_check 10 times, every 2 s. Reconnects after a failure.
PHO_BPS_Calibration_Basic.modx	Calibration with 9 poses (entry routine calibration_main) - see Calibration.
PHO_BPS_Calibration_Expert.modx	Calibration with poses in an array, a check for untaught poses and reduced acceleration (entry routine calibration_main) - see Calibration.

3.2.1 Basic bin picking example

The program connects to the vision controller, initializes one vision system and then repeats these steps: wait for the scan, request the trajectory, move to the Start pose, receive the trajectory, pick and place. The program requests the next scan after the robot leaves the bin, so scanning and planning run while the object is being placed.

Name: PHO_BPS_Basic.modx (folder example_programs)

MODULE PHO_BPS_Basic

    ! Copyright (c) 2026 Zebra Technologies s.r.o.
    ! All rights reserved
    ! Description: Basic binpicking example
    ! Controller: OmniCore, RobotWare version 7.x

    ! ==============================================================================
    !                                 BASIC BIN PICKING
    ! ==============================================================================
    ! This is a basic bin picking template.
    ! Program executes picking in the loop
    ! User is expected to reteach bin picking start and end positions,
    ! set IP Address of Vision Controller and adapt part of the code
    ! responsible for placing and gripper operations in CustomerDefinitions.mod
    ! ==============================================================================

    !ToolData
    TASK PERS tooldata pho_tool:=[TRUE,[[0,0,0],[1,0,0,0]],[0.0,[0,0,0.1],[1,0,0,0],0,0,0]];

    ! Reteach home, start and end binpicking target variables
    VAR robtarget home := [[99999.0, 99999.0, 99999.0], [0, 0, 0, 1], [0, 0, 0, 0], [9E+09, 9E+09, 9E+09, 9E+09, 9E+09, 9E+09]];
    VAR robtarget start_bin_picking_robtarget := [[99999.0, 99999.0, 99999.0], [0, 0, 0, 1], [0, 0, 0, 0], [9E+09, 9E+09, 9E+09, 9E+09, 9E+09, 9E+09]];
    VAR robtarget end_bin_picking_robtarget := [[99999.0, 99999.0, 99999.0], [0, 0, 0, 1], [0, 0, 0, 0], [9E+09, 9E+09, 9E+09, 9E+09, 9E+09, 9E+09]];

    ! Jointtarget placeholders for start and end poses
    VAR jointtarget start_bin_picking_jointtarget;
    VAR jointtarget end_bin_picking_jointtarget;

    ! Vision system ID variable
    VAR num vision_system_id := 1;

    ! =========================== MAIN PROGRAM =====================================
    PROC main()
        ! Clear error code variable
        PHO_ERR_CODE := 0;

        ! Convert robtarget poses to jointtarget format
        start_bin_picking_jointtarget := CalcJointT(start_bin_picking_robtarget, pho_tool);
        end_bin_picking_jointtarget := CalcJointT(end_bin_picking_robtarget, pho_tool);

        ! Connect to Photoneo Vision Controller, change Vision Controller IP address here if needed
        pho_wait_for_server "192.168.1.1", \wait_time:=20, \sys_restart;

        ! Send bin picking initialization request to the Vision Controller,
        ! start and end poses are be used in trajectory planning
        pho_request_init start_bin_picking_jointtarget, end_bin_picking_jointtarget, vision_system_id;

scan_again:
        ! Home pose should be teached out of scanning area
        MoveJ home, v500, fine, pho_tool;

        ! Trigger first scan when in home_pose
        pho_request_scan vision_system_id;

        ! ==================== MAIN LOOP ===========================================
        WHILE true DO

            ! Wait until scanning is completed
            pho_wait_for_scan_completion;

            ! Handle errors if occured
            IF (PHO_OCCURRED_ERR = TRUE) THEN
                err_handling;
                GOTO scan_again;
            ENDIF

            ! Request calculated trajectory
            pho_request_trajectory vision_system_id;

            ! While trajectory is being calculated, move robot to bin picking start position
            MoveAbsJ start_bin_picking_jointtarget, v1000, z200, pho_tool;

            ! Receive requested trajectory (sequence of operations)
            pho_receive_trajectory;

            ! If no object found or other error occured do not proceed to picking
            IF (PHO_OCCURRED_ERR = TRUE) THEN
                err_handling;
                GOTO scan_again;
            ELSE

            ! If trajectory was received sucessfully, Execute bin picking application
            pho_bin_picking;

            ! ==================== PLACING =========================================
            ! Move robot from bin picking end position away from scanning area
            ! - reteach this position for your robot and workcell
            MoveJ [[99999.0, 99999.0, 99999.0], [0, 0, 0, 1], [0, 0, 0, 0], [9E+09, 9E+09, 9E+09, 9E+09, 9E+09, 9E+09]], v500, z30, pho_tool;

            ! Triger next scan and localization,
            ! trajectory for next cycle is calculated while object is being placed
            pho_request_scan vision_system_id;

            ! Commands for actual part placing - reteach positions for your robot and workcell
            MoveJ [[99999.0, 99999.0, 99999.0], [0, 0, 0, 1], [0, 0, 0, 0], [9E+09, 9E+09, 9E+09, 9E+09, 9E+09, 9E+09]], v500, fine, pho_tool;
            gripper_detach;
            MoveJ [[99999.0, 99999.0, 99999.0], [0, 0, 0, 1], [0, 0, 0, 0], [9E+09, 9E+09, 9E+09, 9E+09, 9E+09, 9E+09]], v500, z30, pho_tool;
            ENDIF
        ENDWHILE
    ENDPROC

    ! ==============================================================================
    !                                 ERROR HANDLING
    ! ==============================================================================
    ! Several error situations might occur during bin picking procedure.
    ! Some of them are more serious some are less.
    ! For example if no part is found or trajectory planning fails, program just
    ! tries to repeat the whole sequence. However if communication failure is
    ! detected, program is halted immediately.
    ! It is possible to adapt this behavior if needed.

    PROC err_handling()
        ! Release gripper if needed
        ! gripper_detach;

        ! Move robot to home pose if needed
        !MoveJ home, v500, fine, pho_tool;

        ! Display error
        TPWrite pho_err_info(PHO_ERR_CODE);

        ! If planning failed or no part was found, notify user and continue by next scan (Adapt if needed)
        IF (PHO_ERR_CODE = PHO_PLANNING_FAILED OR PHO_ERR_CODE = PHO_NO_PART_FOUND) THEN
            TPWrite "NO PART FOUND OR PLANNING PATH FAILED";
            pho_request_scan vision_system_id;

        ! If bin picking has not been initialized or service returned error response,
        ! reinitialize and trigger new scan (Adapt if needed)
        ELSEIF (PHO_ERR_CODE = PHO_NOT_INITIALIZED) THEN
            Waittime 5;
            pho_request_init start_bin_picking_jointtarget, end_bin_picking_jointtarget, vision_system_id;

            ! If bin picking service returned error response, reinitialize and trigger new scan (Adapt if needed)
        ELSEIF (PHO_ERR_CODE = PHO_SERVICE_ERR) THEN
            Waittime 10;
            pho_request_init start_bin_picking_jointtarget, end_bin_picking_jointtarget, vision_system_id;

            ! In case of communication failure, rapid program is terminated immediatelly
        ELSEIF (PHO_ERR_CODE = PHO_BAD_DATA OR PHO_ERR_CODE = PHO_TIMEOUT OR PHO_ERR_CODE = PHO_COM_FAILURE) THEN
            ErrLog 4800, "Photoneo internal error", "Communication failure!", "Please reboot the Vision controller"," "," ";
        ELSE
            ErrLog 4801, "Photoneo error", "Communication failure!", "  "," "," ";
        ENDIF

        PHO_OCCURRED_ERR := FALSE;
    ENDPROC
ENDMODULE

3.2.2 Optional utilities

The folder optional_utilities contains modules that are not required for bin picking. You can add them to T_ROB1:

Module	Content
CollisionDetect.modx	pho_bin_picking_with_CD - a replacement for pho_bin_picking that executes the Grasp trajectory (trajectory 2) with collision detection. After a collision, it calls gripper_detach, moves back along the trajectory (pho_execute_back_movement) and sets PHO_OCCURRED_ERR. The module also contains FUNC bool pho_execute_trajectory_with_CD. Requires the ABB Collision Detection option.
CycleTimeChecker.sysx	pho_cycle_time_init starts the measurement. FUNC bool pho_check_cycle_time (call it once per cycle) updates pho_cycle_time and pho_avg_cycle_time (the rolling average of the last 20 cycles). It returns FALSE if the average is above pho_max_allowed_cycle (default 60 s).

3.3 Error handling

After every synchronous request and response receiving procedure, check PHO_OCCURRED_ERR / PHO_ERR_CODE and handle the error in your program. The flags are cleared when the next response is received. Error codes with descriptions and troubleshooting steps are listed in Error codes.

The module defines these error code constants:

RAPID constant	Code	Meaning
PHO_NO_ERR	0	No error
PHO_SERVICE_ERR	1	Service error
PHO_UNKNOWN_REQ	2	Unknown request / unknown operation received
PHO_COM_FAILURE	3	Communication error
PHO_BAD_DATA	4	Bad data
PHO_TIMEOUT	5	Timeout
PHO_BRAND_ID_ERR	7	Brand does not match
PHO_BPS_IN_SIMULATION_MODE	8	Bin picking in simulation mode
PHO_UNKNOWN_ERR	200	Unknown error
PHO_PLANNING_FAILED	201	Path planning failed
PHO_NO_PART_FOUND	202	No object found
PHO_NOT_INITIALIZED	203	Vision system not initialized
PHO_EMPTY_SCENE	218	Empty scene
PHO_UNKNOWN_STATE	219	Unable to read vision system pose
PHO_SCENE_NOT_ACTIVE	220	Scene state not found
PHO_SCENE_ACTIVE	221	Scene state already active
PHO_BBOX_ID_INVALID	222	Invalid bounding box
PHO_WRONG_BP_CONF	255	Wrong bin picking configuration - the module writes event log message 4800 and stops the program
PHO_INVALID_SOL	450	Invalid solution ID
PHO_ANOTHER_SOL	451	Another solution running
PHO_SOL_START_ERR	452	Deployment not started
PHO_SOL_NOT_RUNNING	453	Deployment is not running
PHO_SOL_STOP_ERR	454	Deployment did not stop
PHO_SOL_INCOMPLETE	455	Solution is not complete
PHO_INVALID_BRAND	457	Invalid robot brand
PHO_SOL_RUNNING	458	Solution already running
PHO_SOL_FORBIDDEN	459	Solution access unauthorized
PHO_INVALID_VS	460	Invalid vision system
PHO_CALIB_RUNNING	461	Calibration already running
PHO_CALIB_NOT_RUNNING	462	Calibration not running
PHO_CALIB_STOP_ERR	463	Calibration could not be stopped
PHO_CALIB_START_ERR	464	Calibration could not be started
PHO_CALIB_DISABLED	465	Automatic calibration disabled
PHO_CALIB_NOT_ENOUGH_POINTS	466	Not enough calibration points
PHO_CALIB_NOT_ENOUGH_ACCURACY	467	Calibration accuracy above the allowed threshold
PHO_SCANNER_NOT_AVAILABLE	468	Scanner not available
PHO_CALIB_FORBIDDEN	469	Calibration not allowed

Other codes (for example, calibration point errors 100-134 or 471 No valid license) have no constant. Compare PHO_ERR_CODE with the number directly.

Event log messages written by the module: 4800 - program flow error (pho_receive_trajectory / pho_bin_picking called in the wrong order), wrong bin picking configuration (255) and connection failure with \sys_stop. All of them stop the program. 4801 - RapidSockets blocked by the firewall, or the Robot State Server could not start. Both stop the program.

4 Running the basic bin picking example program

4.1 Prerequisites

Before you run the Basic bin picking example, make sure that:

• A fully configured BPS solution with a single vision system is ready for deployment.

• The robot controller is configured as described in the chapter Robot controller setup.

• The BPS network settings are configured.

• The IP address in pho_wait_for_server and the tool pho_tool match your cell.

• The gripper procedures are implemented (optional - without them, the robot does not actually pick the object).

• The placing poses are taught.

4.2 Bin picking routine execution settings

binpicking_settings in PHO_BPS_Customer_Definitions.modx sets the speed of each trajectory of the bin picking routine in the array pho_speeddata. All 10 values are set to v300 when the module connects. binpicking_settings then overrides the first four:

pho_speeddata{1} := v1000;  ! Approach trajectory by default
pho_speeddata{2} := v500;   ! Grasp trajectory by default
pho_speeddata{3} := v500;   ! Deapproach trajectory by default
pho_speeddata{4} := v1000;  ! End trajectory by default

Adapt the values to your application. If your Grasping method has more trajectories, set more values. The first value applies to the first trajectory of the routine, the second value to the second trajectory, and so on.

4.3 Reteach the robot poses

Teach the home pose, the Start pose and the End pose. The home pose must be outside the scanning area. In the Start pose, the gripper should be approximately above the center of the bin. The End pose can be similar to the Start pose or slightly shifted towards the placing area. Do not place the End pose too far from the bin. This increases the planning time and can cause planning errors.

The basic example uses these variables:

• home - home pose

• start_bin_picking_robtarget - Start pose

• end_bin_picking_robtarget - End pose

• the three MoveJ targets in the placing part

The program converts the Start and End poses to joint targets with CalcJointT and passes them to the Initialization request:

start_bin_picking_jointtarget := CalcJointT(start_bin_picking_robtarget, pho_tool);

4.4 Runtime

Deploy your BPS solution.

Open the Code application on the FlexPendant and select PP to Main:

Choose AUTO or MANUAL mode and adapt the speed override if needed. Then press Play (AUTO) or Enable + Play (MANUAL).

The Action Request Client status on the Deployment page changes to CONNECTED. The robot starts sending requests and executing bin picking movements.

NOTE: Be ready to stop the robot immediately. We strongly recommend reducing the speed to 10% of the maximum during the first bin picking tests.

5 Robot State Server

The Robot State Server (PHO_BPS_State_Server.modx, routine pho_state_server_main) runs in the background task PHO_STATE_SERVER. It starts automatically with the controller and works independently of the main program.

• It listens on port 11004 on the IP address of the controller’s public network - the WAN port (GetSysInfo(\LanIp)).

• If the automatic address is wrong (OmniCore has a different port layout than IRC5), set the PERS string sPho_IP_StateServer in PHO_BPS_State_Server.modx to the IP address of the correct interface (for example "192.168.125.1"). Empty = automatic.

• After the vision controller connects, it sends the brand ID. Then, every 0.1 s (10 Hz), it sends two messages: the joint states (6 axes, rad) and the tool pose (tool0 in wobj0 - X, Y, Z in mm and a quaternion) of the task robot_task_id (default T_ROB1).

• It pauses while pho_wait_for_server connects (flag pho_connecting_to_server).

• If the connection is lost, it writes PhotoneoStateServer - connection lost, restarts and waits for a new connection. While it waits, it writes Waiting for connection….

• Before it starts, it checks the firewall (see Firewall configuration). If the server socket cannot be created (for example, because sPho_IP_StateServer is wrong), it writes event log message 4801 Photoneo State Server startup failed and stops.

The vision controller uses this data for visualization and for the robot pose in hand-eye scanning. The message format is described in Robot communication overview.

6 Calibration

Two calibration example programs are provided. Both use the entry routine calibration_main. Set it as the entry point (or call it from main). The first time, run it step by step in MANUAL mode.

The calibration modules declare their own pho_tool and vision_system_id. Load a calibration module into T_ROB1 instead of the bin picking example.

PHO_BPS_Calibration_Basic.modx works as follows:

1. Connects with pho_wait_for_server and moves to home_pose.

2. If auto_calibration is TRUE (default), starts the calibration: pho_request_calib_start(solution_id, vision_system_id).

3. For each of the 9 poses calib_pose_1 … calib_pose_9: moves there (fine), waits 1 s and calls pho_request_calib_add_point. The result of each point is written to the FlexPendant.

4. Moves back to home_pose.

5. If auto_calibration is TRUE, saves the result with pho_request_calib_save (the accuracy is stored in pho_calib_acc [µm] and the camera pose in pho_camera_pose), then stops the calibration with pho_request_calib_stop.

With auto_calibration = FALSE, the calibration is semi-automatic: start the calibration in BPS, run the program to add the points, and then check and save the result in BPS.

If the accuracy is not good enough, pho_request_calib_save returns 467 (PHO_CALIB_NOT_ENOUGH_ACCURACY) and the result is not saved. Use poses with more variation in position and rotation, and repeat the calibration.

PHO_BPS_Calibration_Expert.modx does the same with the poses in the array calib_poses{9}. It also:

• checks that all poses were retaught (event log message 4804 for each untaught pose, then stops at these poses so that you can teach them),

• limits the acceleration with WorldAccLim (acc_decc_ramp),

• waits wait_before_scan seconds at each pose and logs each point (4800 error / 4801 info),

• has auto_calibration = FALSE by default.

For details on the calibration requests, see Action requests.

7 Troubleshooting

The module handles communication problems as follows:

• Connection fails - pho_wait_for_server sets PHO_ERR_CODE to 5 (timeout) or 3 (connection closed), then acts according to \sys_stop / \sys_restart (see Connection procedures).

• Socket timeout or closed socket while sending or receiving - the module sets PHO_ERR_CODE to 5 or 3 and PHO_OCCURRED_ERR to TRUE and returns. The program continues, so your program must handle the error. Reconnect with pho_wait_for_server and send the Initialization request again.

• Response does not match the request, or trajectory data fail the checksum - PHO_ERR_CODE = 4 (Bad data) and the rest of the message is discarded.

• While PHO_ERR_CODE is 3, 4 or 5, pho_wait_for_scan_completion and pho_receive_trajectory return immediately. Clear PHO_ERR_CODE (or reconnect) before the next cycle.

Problem	What to check
FlexPendant shows Not able to connect to vision controller	IP address in pho_wait_for_server, network cable and subnet, deployed solution in BPS.
Event log 4801 Communication not enabled in the robot firewall!	Enable RapidSockets in the Firewall Manager for the network shown in the message and restart the controller.
Event log 4801 Photoneo State Server startup failed	sPho_IP_StateServer must be empty or an IP address of the controller. Port 11004 must be free.
State Server stays at Waiting for connection…	The PHO_STATE_SERVER task is running, PHO_BPS_BASE.sysx is loaded in it, and the vision controller can reach the controller’s WAN IP on port 11004.
Event log 4800 Program flow error	pho_receive_trajectory was called without pho_request_trajectory, or pho_bin_picking was called without a successful pho_receive_trajectory.
Event log 4800 WRONG BP CONFIGURATION	Error 255 - check the BPS console.
Error 7 or 457 (brand)	The Robot module version does not match the Bin Picking Studio version - use Robot module 1.12 with Bin Picking Studio 1.12.
Robot does not move after pho_receive_trajectory	Check PHO_ERR_CODE - for example 201 (planning failed) or 202 (no object found).
Need more information	Call pho_enable_debug after pho_wait_for_server. The module then writes every request and response to the FlexPendant.

8 Version and brand ID

• Robot module version: 1.12.0 (shown in the header of PHO_BPS_BASE.sysx).

• Brand ID: ABB_IRB/1.12.0XXXXXXXXXX - constant robotIdStr in PHO_BPS_BASE.sysx. The module sends it every time the Action Request Client or the Robot State Server connects. BPS uses it to detect the robot brand and the module version. Do not change it.

• Default ports: Action Request Server 11003 (vision controller), Robot State Server 11004 (robot controller).

See Robot module support matrix for all brands and versions.

9 Migration

9.1 BPS 1.11.x -> BPS 1.12.x

Replace PHO_BPS_BASE.sysx and PHO_BPS_State_Server.modx with the new versions and restart the controller. Robot programs do not need any changes, because the module handles the new Info data format internally.

New in 1.12: the variables pho_nn_label, pho_max_z_height and pho_tilt, and the example program PHO_BPS_Comm_Check.modx.

The OmniCore module 1.12 is based on the IRC5 module 1.12. If you update from an older OmniCore module, remove all old Photoneo modules and install the new ones as described in Robot module installation.

For details, see BPS migration guide 1.11.x -> 1.12.x.

9.2 Robot module update

1. Back up your customer definitions module - it contains your gripper procedures and speed settings.

2. Remove the PHO_STATE_SERVER task, the automatic loading entries and the old modules. Then restart the controller.

3. Install the new modules as described in Robot module installation.

4. Copy your changes from the old customer definitions module to PHO_BPS_Customer_Definitions.modx.

5. Check the API changes listed above and update your main program if needed.